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8.4.4 P-Values 

In the above discussions, we only reported an "accept" or a "reject" decision as the conclusion 
of a hypothesis test. However, we can provide more information using what we call P-values. 
In other words, we could indicate how close the decision was. More specifically, suppose we 
end up rejecting Hq at at significance level Ot = 0 . 05 . Then we could ask: "How about if 
we require significance level Ct = 0 . 01 ?" Can we still reject More specifically, we can 
ask the following question: 

What is the lowest significance level CX that results in rejecting the null hypothesis? 

The answer to the above question is called the P-value. 

P-value is the lowest significance level Ct that results in rejecting the null hypothesis. 
Intuitively, if the P-value is small, it means that the observed data is very unlikely to have 
occurred under Ho , so we are more confident in rejecting the null hypothesis. How do we 
find P-values? Let's look at an example. 


Example 8.29 

You have a coin and you would like to check whether it is fair or biased. More specifically, let 
9 be the probability of heads, 6 — P(H). Suppose that you need to choose between the 
following hypotheses: 


Hq (the null hypothesis): The coin is fair, i.e., 9 — Oo — ^ ■ 

H\ (the alternative hypothesis): The coin is not fair, i.e., 9 > 

We toss the coin 100 times and observe 60 heads. 

1. Can we reject Ho at significance level Ct = 0 . 05 ? 

2. Can we reject Hq at significance level Ct = 0 . 01 ? 

3. What is the P-value? 

• Solution 

o Let X be the random variable showing the number of observed heads. In our 
experiment, we observed X — 60 . Since 71 — 100 is relatively large, 
assuming Ho is true, the random variable 

X-n9 0 X - 50 

W — — — =- 

y/n9 0 (l - 9o) 5 
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is (approximately) a standard nonnal random variable, iV(0, 1). If Hq is true, 
we expect X to be close to 50, while if -f^i is true, we expect X to be larger. 
Thus, we can suggest the following test: We choose a threshold C. If W < C, we 
accept Hq; otherwise, we accept H\. To calculate P(type I error), we can 
write 

P (type I error) = P(Reject Hq \ Hq) 

= P(W > c | H 0 ). 

Since W ~ N (0, 1) under Hq , we need to choose 

C = Z ai 

to ensure significance level ct. In this example, we obtain 


X — 50 60- 50 



1. If we require significance level Ct = 0.05, then 

C = ZQ.Q5 = 1.645 

The above value can be obtained in MATLAB using the following 
command: norminv ( 1 — 0.05) . Since we have W — 2 > 1.645, 
we reject Hq, and accept H\. 

2. If we require significance level a — 0.01, then 

c — zq _ q ^ — 2.33 

The above value can be obtained in MATLAB using the following 
command: norminv (1 — 0.01) . Since we have W — 2 < 2.33, 
we fail to reject Hq, so we accept Hq. 

3. -P-value is the lowest significance level OL that results in rejecting Hq. 
Here, since W — 2, we will reject Hq if and only if c < 2. Note that 
Z a = C, thus 

a = 1 — 4>(c). 

If C = 2, we obtain 

a = 1 - $(2) = 0.023 

Therefore, we reject Hq for a > 0.023. Thus, the P- value is equal to 

0.023. 
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The above example suggests the following way to compute .P-values: 

Computin g P-Values 


Consider a hypothesis test for choosing between Hq and H\. Let W be the test statistic, and 
W\ be the observed value of W. 

1. Assume H 0 is true. 

2. The P’-value is -P(type I error) when the test threshold C is chosen to be C = W\ . 
To see how we can use the above method, again consider Example 8.29 . Here, 


w = 


X — 50 


which is approximately iV(0, 1) under Hq. The observed value of W is 

60-50 


W\ 


= 2 . 


Thus, 


P — value = P (type I error when c = 2) 
= P{W > 2) 

= 1 - $(2) = 0.023 


<— previous 
next —> 
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